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Introduction 


Students’ perceptions of their world, their understanding of and attitudes toward others, and their view of 
self are greatly influenced daily by an educator’s selection of instructional materials. Classroom curricular 
resources often reflect stereotypes and biases about diverse groups of students, personal identities, and 
nontraditional family structures perpetuated in our society. Students are expected to accept the 
information presented as accurate and valid. “Therefore, it is very important that teachers [and other 
educators] select instructional materials that do not promote existing stereotypes, create new ones, or 
prevent students from acquiring accurate and valid information” (Ferguson, 2015, p.1). 


By ensuring young people access learning materials that expose them to diverse images and narratives, 
school teams will support students of varying backgrounds in developing a healthier self-concept and 
greater self-esteem. This has the potential to create a balanced learning environment that addresses or 
corrects pervasive misconceptions. Additionally, promoting instructional materials that value or prioritize 
diversity allows students to see themselves as a part of the learning cycle and contributes to creating an 
inclusive community, a critical step in promoting social justice and equity in a school setting. To accomplish 
this goal, educators need the tools and guidance to select materials that provide both a balanced and 
accurate perspective of diversity among individuals and communities, as well as correct or reduce 
misconceptions about others. 


Many approaches are being used to reduce implicit and explicit bias, yet little is still known about their 
relative effectiveness. Studies of both children and adults reveal that proximity and the exposure to more 
positive, counter-stereotypical images of people from different racial and ethnic groups can have some 
effect on decreasing implicit negative associations. According to Gonzalez (2016), youth as young as 10 can 
internalize positive associations to counteract the stereotypical associations they may already have. In this 
study, researchers were most concerned with racial bias and used counter-stereotypical vignettes and 
images as an intervention, measuring Implicit Association Test (IAT) scores before and after exposure. 
Allowing young people to engage individuals who contrast with usual stereotypes promoted an 
understanding of the range of types and subtypes within a specific racial group and decreased implicit 
preferences. Additionally, the National Equity Project (Osta & Vasquez, 2019) suggests that attempts to 
dismantle bias alone are not sufficient and emphasize the importance of also exposing structural 
inequalities that perpetuate implicit bias. Therefore, a windows and mirrors approach is vital to ensure 
curricular materials are equitable. 


Students seeing their identities mirrored in their instructional materials has been shown to promote 
positive social identity development by increasing their pride, confidence, and healthy self-esteem. In 
addition, a mirrors approach supports their recognition of the distinctions in the traits of the dominant 
culture, their home culture, and other cultures (Scharf, 2018). Through a windows approach, students are 
exposed to the lived experiences of others, expanding their understanding and empathy while exploring 
diverse social, cultural, political, and historical contexts. 
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Purpose and Overview 


The purpose of this report is to support the Michigan Department of Education (MDE) in building the 
capacity of local educational agencies and schools in the identification of resources and tools that can be 
used by instructional teams to evaluate and assess bias in curricular materials. The materials reviewed 
provide insights into critical questions or approaches that ensure students will encounter materials that 
include and reflect a diversity of perspectives, narratives, and histories while elevating the contributions of 
non-White, non-male, non-dis/abled, or non-cisgender individuals. This report unpacks selected findings 
from a scan of available resources. A sub-focus on distinguishing between the different types of evaluation 
structures provides insight into whether the available resource is a tool (can be used as it stands) or 
supportive resource (is instructional and informative to the evaluation process). For each resource, we also 
provide an overview of major highlights to support the MDE team in understanding high-level features and 
notable factors that might inform usability. Finally, we offer considerations for the MDE based on an 
analysis of these resources. 


Methodology 


To complete this scan, we worked closely with the MDE to identify the following focus: to find tools or 
resources for evaluating bias in instructional materials. After identifying the focus, we explored a variety of 
resources, research, and other publicly available materials focused on evaluating bias. To guide and focus 
our review, we developed an inquiry framework, looking specifically to answer the following questions: 


How can the resource be categorized? 

What is its origin/major characteristics? 

How can it be used? 

What are the key considerations of usage? 

What is the depth and breadth of bias presented in the resource? 


We entered information into a data collection template, reviewing and screening a total of 27 potential 
resources. Additionally, we validated our evaluation of each resource using team reads and consensus- 
building to determine highlights, strengths, and usefulness. This report synthesizes the reviews, focusing 
on the 14 resources that are most compelling or appear best aligned to the focus of this scan. 
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Summary of Tools 


The following tools can support instructional teams in evaluating bias in instructional materials. 


Tool Name: Culturally Responsive Curriculum Scorecard 


Overview: This tool was designed to support parents, teachers, students, and community members in 
assessing whether the English Language Arts curriculum for their schools is culturally responsive. The 
scorecard begins by clarifying the definition and significance of Culturally Responsive Education (CRE). The 
authors provide tips for completing the analysis in The Seven Steps to Complete Your Curriculum Scorecard. 
There are three sections of the scorecard: representation, social justice, and teachers’ materials. The 
Representation section focuses on diversity of characters and authors, and accurate portrayals. The Social 
Justice section focuses on decolonization, power and privilege, centering multiple perspectives, and 
connecting learning to real life. The Teachers’ Materials section assesses the guidance provided by the 
teacher on nine essential categories including, but not limited to bias awareness; engaging students in 
culturally sensitive activities; and customizing and supplementing the curriculum as needed to reflect the 
interests, backgrounds, and diverse perspectives of the student population, among other items. A scale 
follows, which outlines ratings from Not Satisfied to Very Satisfied. There is detailed guidance provided on 
how to interpret the score. There is a partner toolkit which provides guidance and resources to support 
educators, administrators, communities, students, and parents on the next steps to building culturally 
responsive classrooms and schools. 

Considerations: The scorecard has clear guidance and uses most current language. The scorecard also 
includes a partner toolkit to support local implementation. The scorecard is designed so it can be 
customized to the context of a district. The tool does not define teaching materials and requires a level of 
critical consciousness and an understanding of issues related to educational equity, such as culture, 
diversity, and inclusion. Pairing this with professional learning and utilizing the partner toolkit would be an 
important component to support fidelity of implementation. 

Tool Release Date: 2019 

Citation: Bryan-Gooden, J., Hester, M., & Peoples, L. Q. (2019). Culturally responsive curriculum scorecard. 
New York: Metropolitan Center for Research on Equity and the Transformation of Schools, New York 
University. 


Tool Name: Assessing Bias in Standards and Curricular Materials 


Overview: The tool enables users to consider equity-oriented domains, articulated through a set of rubrics 
and indicators, to surface biases within both standard development and interpretations, as well as 
curricular material creation, selection, and application. The tool assesses a wide range of biases, such as 
linguistic, invisibility, tokenism, and representation, and each is well-defined. A set of directions on how to 
use the rubrics, along with a scoring and analysis guide are provided to assist in the evaluation process. 
Users of the tool are encouraged to engage in critical reflection and dialogue around a set of questions 
about the purpose of the curriculum and the nature of students prior to beginning the review process. 
Considerations: The set of rubrics and indicators organized around a set of equity-centered domains and 
the supporting scoring and analysis guide are this tool’s major strengths. However, the tool assumes the 
user possesses a sophisticated understanding of critical consciousness and issues related to educational 
equity. This tool would require technical assistance and conversations regarding topics and terminology to 
ensure understanding and appropriate use of the tool. 

Tool Release Date: 2017 

Citation: Coomer, M. N., Skelton, S. M., Kyser, T. S., Thorius, K. A. K., & Warren, C. (2017). Assessing bias in 
standards and curricular materials. Equity Tool. Indianapolis, IN: Great Lakes Equity Center. 
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Tool Name: Washington Models for the Evaluation of Bias Content in Instructional Materials 


Overview: The resource seems to include three different tools or guidelines that can be used to consider 
bias when selecting instructional materials. The first tool, titled Ten Quick Ways to Analyze Children’s Books 
for Racism and Sexism, was adapted from the Council on Interracial Books for Children’s Guidelines for 
Selecting Bias-Free Textbooks and Storybooks, first published in 1980. The original tool was described as 
one of the first used to evaluate books for bias but has been out of date for several years. The tool lists 10 
categories (e.g., illustrations, loaded language) and includes questions to ask for each category when 
reviewing children’s books. The second tool is a one-page table that lists six examples and non-examples to 
consider when reviewing for bias. The third tool is titled General Criteria for Evaluating Instructional 
Materials. This four-paged tool includes Likert-scaled items to be evaluated based on the following 
categories: Gender/Sex, Multicultural, Persons with Disabilities, and Socio-Economic Status. Items are to be 
rated on ascale from 3 to N/A; however, the scaling is difficult to follow, and directions are not included. 
An example item is: Stereotyping language is avoided. The user would rate the item as, 3=Standard is 
clearly articulated or inferred; 2=Standard is present, but limited; 1=Limited presentation of standard; or, 
N/A. It is not clear how one would rate this and if a low or high score is ideal for each category. 
Considerations: The introduction encourages district users to include parents on the committee when 
evaluating curricular material for bias and requires Washington State school districts to have a process 
when there are complaints about the curriculum (cites specific Washington state law). It seems that the 
author found or created three different tools and put them into one document with minimal explanation or 
guidance for the users. One tool includes a reference, but two of the tools are not cited. The document 
references laws specific to Washington state and includes them in the appendix. At times, the document 
uses the term “minority” to lump marginalized groups together. 

Tool Release Date: 2009 

Citation: Washington Office of Superintendent of Public Instruction. (2009). Washington models for the 
evaluation of bias content in instructional materials. Olympia, WA. 


Tool Name: Evaluating American Indian Materials and Resources for the Classroom 


Overview: This tool assists teachers, librarians, and curriculum directors in the analysis and evaluation of 
a wide variety of written and visual instructional materials and resources (including textbooks and 
literature; author/illustrator; accompanying and supplementary materials; illustrations, visual data, and 
maps; DVDs and film; and web sites and online content) for anti-Indian biases. To ensure American Indian 
topics are treated fairly, objectively, and accurately, educators are guided to use non-biased terms when 
referring to indigenous inhabitants, and to weed out instructional materials that distort the history and/or 
distort the culture and identities of the American Indians. The user is provided information about the 
significance of some terms historically used to refer to American Indians. Several forms of bias are 
introduced and defined, such as tokenism, selectivity, omission, and biased language. Along with each form 
of bias are descriptions of how specific forms of bias show up in the different types of instructional 
materials listed above. A separate section is devoted to assessing bias in historical and primary documents 
as these are believed to reflect the social, cultural, political, and historical context in which they were 
created and can be effectively used to examine how such biases have or could influence policies, people’s 
lives, and institutions. In addition to a series of guiding questions, there are two assessment tools to assist 
educators in assessing instructional materials for the biases. A list of resources to aid in the evaluation and 
identification of anti-bias materials is also provided. 

Considerations: The tools and guiding questions provided assist educators in proactively addressing two 
historically common broad anti-Indian biases: distortions of history, and distortions of culture and identity. 
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Another is the comprehensive discussion of how anti-Indian bias cuts across different types of instructional 
materials and resources. The description of how anti-Indian biases show up in a wide variety of written 
and visual instructional materials and resources is quite informative even for those assessing for 
stereotypes and biases toward other cultural groups or identities. However, this document focuses solely 
on the American Indian. One section of the resource refers specifically to the Montana Indians. 

Tool Release Date: Original 1992, Revised 2015 

Citation: Ferguson, L. (2015). Evaluating American Indian materials and resources for the classroom. 
Helena, MT: Montana Office of Public Instruction. 


Tool Name: Edgenuity: Curriculum Development Guidelines on Bias and Sensitivity 


Overview: This tool assists educators in determining whether instructional materials provide all students 
equal opportunity to demonstrate their abilities and knowledge. One underlying premise of this tool is that 
“instructional material may be biased if the language or content is differentially familiar to subgroups of 
students or if document structure or format is differentially difficult for subgroups of students” (p. 3). A 
second premise is that instructional materials may include stereotypes and inadequate or unfavorable 
representations of specific groups of students, making it more difficult for some students to process the 
content leading to lower academic performance. Educators are provided with a brief definition and a few 
examples of biases and stereotypes to give them some understanding of the underlying purpose of the two 
checklists of questions to determine if the instructional materials and assessments include stereotypes 
and/or specific biases. The list of questions for reviewing for bias is much more extensive than the list for 
stereotypes. 

Considerations: The strongest aspect of this resource is the list of questions provided for determining if 
the instructional material contains forms of bias that will disadvantage some students’ academic 
performance. However, this tool depends on a sophisticated understanding, by the user, of the cultural 
backgrounds of the students engaging in the curriculum, and the terms commonly used in the locale in 
which the curriculum is situated. It focuses the user's attention only on the subgroups who will engage with 
the materials. The reviewer could unintentionally overlook and thus perpetuate a stereotype or bias ofa 
group of diverse cultures or identities not within the user's immediate context. 

Tool Release Date: n.d. 

Citation: Edgenuity, Inc. (n.d.). Curriculum Development Guidelines on Bias and Sensitivity. Scottsdale, AZ. 


Tool Name: Bias Evaluation Instrument 

Overview: The Bias Evaluation instrument was developed by the Novia Scotia Department of Education. It 
is based on Ontario Curriculum Centre’s tool The Bias Assessor (1998). It is based on the premise that every 
individual has bias and, therefore, must evaluate selected materials to ensure that they are inclusive of 
broader diversity and not solely influenced by the personal and social identities, values, and experiences of 
the selector. The document includes a tool with items to rate bias in curriculum regarding the following 
seven categories: Appearance, Belief System, Ability/Disability, Family Structure, Gender, Race and 
Ethnocultural, and Socioeconomic Status. The document starts with a Preamble and includes Principles of 
Learning that outline what teachers and administrators should consider when establishing learning 
environments and designing instructional activities. The learning principles are based on Social 
Constructivist theory and Culturally Responsive instructional practices. The document includes directions 
on how to use the tool and important tips to consider when assessing resources for bias. Each of the seven 
categories are defined and provide specific examples of what learning materials should include regarding 
that category. The rating scale includes questions, and the rater can check yes, no, or n/a. The rater would 
then determine if it is recommended, needs revision, or is unacceptable for each category. The tool also 
provides a brief set of considerations to help the reviewer understand what a good review process looks 
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like and what posture they should take during review. These include reading (a) the instructional material 
in entirety; (b) looking for both subtle and overt biases, fragmented representations, or tokenism; and (c) 
working in collaboration to review materials specifically where the reviewer might have limited familiarity 
with particular areas of diversity. 

Considerations: The tool is succinct but also covers a wide range of diversity categories. Additionally, the 
document is specific to the Canadian context and uses the British spelling of terms such as “behaviour” and 
“colour.” 

Tool Release Date: 2001 

Citation: Nova Scotia Department of Education. (2001). Bias evaluation instrument. Halifax, NS, Canada. 


Summary of Supportive Guidance 


The following resources can support instructional teams in preparing their educators for the process of 
evaluating bias in instructional materials. These supplementary materials can support professional 
development to aid the process of eliminating bias from instructional materials. 


Tool Name: Fairness, Bias, and Cultural-Responsiveness Checklist for Assessments 
Overview: The checklist is used to evaluate assessments or other performance tasks for bias, fairness, and 


cultural responsiveness. The evaluation tool is organized into five categories: Bias, Stereotyping, Fairness, 
Cultural Responsiveness, and Controversial Topics. The first two categories contain checklist items used to 
eliminate bias and stereotyping from an assessment. The last three categories contain checklist items used 
to expand one’s thinking regarding fairness and cultural responsiveness in curriculum and instructional 
practices as they connect to performance tasks. Raters evaluate each criteria item with a Yes or No, but also 
have space to include qualitative explanations for their ratings, as well as recommended revisions. The 
document includes a student-centered perspective, considers Universal Design for Learning (UDL), and 
culturally responsive practices. The resources used to create the document include literature on bias in 
assessments and culturally responsive education. 

Considerations: This resource can be a stand-alone document to evaluate assessments for bias, or it can be 
used in conjunction with another tool, the Assessment Validation Checklist, for a comprehensive evaluation 
of student assessments. It is short, user-friendly, and could be used to engage a group in dialogue about 
bias in assessments. However, the checklist is not a comprehensive evaluation tool for curriculum but is for 
assessments only. Various diversity categories are mentioned but not expanded upon. 

Tool Release Date: 2017 

Citation: Center for Collaborative Education. (2017). Fairness, bias, and cultural-responsiveness checklist for 
assessments. Boston, MA. 


Tool Name: Critical Practices for Anti-bias Education 

Overview: The document is not a tool but a framework and thorough description of the essential features 
of anti-bias education. The first four sections of the document are organized into the following categories: 
Instruction, Classroom Culture, Family and Community Engagement, and Teacher Leadership. The fifth 
section is the Anti-bias Framework, which informs the first four sections. The first four sections include 
recommended practices, connections to anti-bias education, and strategies. The critical practices are based 
on the values exemplified in the Teaching Tolerance Anti-bias Framework (section 5). The Framework is a 
roadmap for anti-bias education at every grade level and is organized into four domains: Identity, Diversity, 
Justice, and Action. The domains represent a continuum of engagement in anti-bias, multicultural, and 
social justice education. The Framework includes a set of anchor standards, corresponding grade-level 
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outcomes, and school-based scenarios to show what anti-bias attitudes and behavior may look like in the 
classroom. 

Considerations: Authored by a reputable and well-known organization that provides free technical 
assistance resources that can be reproduced. The Framework is based on critical theory, transformative 
education, culturally responsive and anti-racist education, and social justice educational practices. 
Although it is thorough, the 36-page document may be long and cumbersome for someone looking for a 
tool to evaluate the curriculum. However, the document may be best supported with technical assistance, 
so itis broken up in smaller portions and discussed. The document utilized Common Core State Standards 
when it was created, which may have political implications that could be positive or negative. 

Tool Release Date: 2018 

Citation: Scharf, A. (2018). Critical practices for anti-bias education. Teaching Tolerance. 


Tool Name: Guidelines for Identifying Bias in Curriculum and Materials 


Overview: The document is a set of guidelines, not a tool, that juxtapositions a non-example to an example 
when considering bias and stereotyping that may exist in the curriculum. It includes three categories of 
examples: (1) Stereotypes; (2) Language; (3) Omission, Exclusion, and Perspective. 

Considerations: The guidelines provide good examples and alternative considerations that may be used to 
help identity bias. The document expands beyond traditional diversity categories to include family 
structure, occupation, and body shape/size. The authors state that the examples are not meant to be 
exhaustive but are a starting point to begin conversations. The short introduction describes the importance 
of cultural pluralism in our schools/society and gives a rationale for the need to review the curriculum for 
bias and stereotypes. However, the layout is difficult to read, the print is very small, and is not visually 
accessible. 


Tool Release Date: 2003, but adapted from another resource published in 1992 
Citation: The Safe School Coalition. (2003). Guidelines for identifying bias in curriculum and materials. 
Seattle, WA. 


Tool Name: Guide for Selecting Anti-Bias Children’s Books 
Overview: This guide assists teachers and families in the evaluation of books for accurate and respectful 


messages about diversity, power relationships among different people, and various social identities (e.g., 
racial, ethnic, gender, economic class, sexual orientation, and dis/ability). Though the primary focus is on 
identifying the biases, stereotypes, omissions, and misrepresentations in the visual (illustrations) and 
verbal messages embedded in books, there are some guidelines for taking quality into account. This is to 
ensure the books engage the child’s attention. A range of forms of bias are explored in the guide to avoid 
“undermining children’s sense of self, positive attitudes toward others, and motivations to act fairly” 
(Derman-Sparks, 2016). A definition for each form of bias is provided and its potential impact on the child 
is described. In some instances, examples are provided to illustrate the different ways in which the bias 
may show up, along with questions to guide the reviewer's assessment of the book. In recognition that 
everyone has been unconsciously acculturated into prejudicial and stereotypical thinking, the user is 
provided with a series of guiding questions that should surface recognition of biases embedded within a 
book. 

Considerations: The different forms of bias identified reflect many of the attitudes held in modern society 
about diversity, power relationships among different groups of people and various identities. This resource 
addresses bias against a wide range of children’s identities. The forms of bias described in this resource are 
quite inclusive of those often overlooked. This resource provides an easy-to-follow list of questions to 
quickly guide one in the selection of anti-bias children’s books in a manner that should help users recognize 
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their own-held stereotypes. However, the guide is specific to young children’s books. Although the guide 
provides examples and some discussion of each form of bias, it does seem to assume that the user has some 
sensitivity to the stereotypes of and biases towards diverse populations. 

Tool Release Date: 2013 

Citation: Derman-Sparks, L. (2016, April 14). Guide for selecting anti-bias children's books. Washington, DC: 
Teaching for Change. https://www.teachingforchange.org/selecting-anti-bias-books 


Tool Name: Creating an Anti-Bias Learning Environment 
Overview: Provides a checklist and prompts for assessing how effective an educator or school is in 


promoting a bias-free environment. This resource provides 11 tips for teachers preparing to raise issues of 
diversity and bias in the classroom to consider prior to utilizing the checklist to evaluate their classroom or 
school’s multicultural efficacy. There are two parts to the checklist, one for teachers to utilize to assess 
their classroom and instructional materials and another for educators to assess how effective the school is 
in creating a bias-free learning environment. There are three possible ratings for each item in the checklist: 
I/We haven't thought about this, I/We need to do this better, or I/We do this well. Once completed, educators 
have a clear view of the areas in which they need to improve or need to consider incorporating into their 
learning environment. 

Considerations: This resource can be used as a self-assessment for educators to determine the 
effectiveness of their classroom as it relates to diversity, equity, and inclusion. This could also serve as a 
resource for school leadership teams to assess instructional materials more broadly. While this resource 
does provide general tips to consider when addressing bias and diversity in the classroom, it would require 
additional training to be of greater use to educators. 

Tool Release Date: 2012 


Citation: Anti-Defamation League. (2012). Creating an anti-bias learning environment. New York. 


Tool Name: Minimizing Bias in Assessing Student Work 
Overview: This resource provides guidance on ensuring practitioners are aware of how bias affects scoring 


in assessments that require a qualitative response. The resource focuses mostly on perspective and how a 
scorer perspective and positionality can impact how they might engage written responses. It contends that 
assessment responses are a reflection of how an individual identifies and to which communities they 
belong. As schools serve more and more diverse populations, scorers need tools to counteract potential 
bias. The text suggests that scorers use two key cognitive development strategies: (1) assessing depth of 
argument and ability of a student to discuss an issue from a variety of viewpoints, and (2) assessing the 
degree to which the response considers systemic factors and moves beyond personal and individualized 
opinions. Additionally, it calls out strategies related to knowledge and argument structure. Scorers can 
focus on an argument's structural coherence or evaluate the quality and complexity of the argument itself. 
Considerations: The resource provides clear guidance that is situated within theoretical frameworks. This 
guidance includes very practical examples. However, this resource is guidance only and does not include 
any corresponding tools. The guidance is also focused on the scoring of the assessment but not on the ways 
in which assessments can be biased. 

Tool Release Date: 2017 

Citation: Steinke, P., & Fitch, P. (2017). Minimizing bias when assessing student work. Research & Practice 
in Assessment, 12, 87-95. 
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Tool Name: Standards for Evaluating Instructional Materials for Social Content 


Overview: The standards were published in California per state law requiring that instructional material 
accurately and equitably portray diversity within the United States. They were designed to provide 
evaluators and practitioners with a standards-based framework to ensure that instructional materials 
contribute to a positive learning experience for all students. The resource focuses on depicting individuals 
related to race, ethnicity, gender, age, ability, and religion. Each standard includes a section on purpose, 
method, and applicability. Method provides a high-level overview of what is expected to be measured for 
each standard. Applicability sections break out guidance on portrayals within identity categories in areas 
such as culture, practices, achievement, references or labels, or activities. Some diversity categories are 
presented with an intersectional focus (i.e., the pervasive interchangeability of minority and low income). 
In addition to common categories, the standards also address labor functions, environmental issues, 
substances, humane treatment of animals and people, the constitution, preventing unnecessary brand 
advertisements, and promoting nutrition and physical activity. 

Considerations: This resource is comprehensive and tackles the complexity of intersectionality in addition 
to key categories. It provides both depth in each category and a high-level overview of how and why the 
standard exists. The language used is slightly outdated and could be adjusted with race and gender identity 
categories. The resource is also heavy on guidance and would require some support to apply. This could 
include professional learning, creating an auxiliary instrument, and modeling the application of standards 
to an instructional resource. 

Tool Release Date: 2013 

Citation: California Department of Education. (2013). Standards for evaluating instructional materials for 
social content. 


Tool Name: See, Hear, and Speak No Evil: A Content Approach to Evaluating Multicultural 


Multimedia Materials 

Overview: This resource uses a conceptual approach to the evaluation of multimedia materials. It 
identifies four (4) evaluation components: content objectivity, language use, subject mastery, and 
resources. The guidance shares what characteristics should be considered in the selection of 

materials. Across each criterion, the resource notes which types of medium are to be reviewed (i.e., text, 
graphics, sound, etc.) and essential evaluation principles. While this information is presented in a chart, the 
resource also contains deeper narrative to help the user understand how to evaluate against the criteria. 
All criteria are defined and summarized with a focus on the depth and nuance contained therein. For 
instance, content objectivity breaks out bias across race, culture, ethnicity, gender, sexual orientation, and 
age as well as provides five additional sub-criteria to review within objectivity. These include imbalance 
and selectivity, unrealistic and misrepresentation, invisibility, fragmentation and isolation, and 
stereotyping. 

Considerations: This resource is grounded in research and provides a depth of guidance across criteria. 
This depth considers diversity across several different elements. It is purposefully conceptual in a way that 
makes it potentially ideal for professional development but not necessarily useful to pick up and apply in 
an evaluative process. It could help individuals learn and understand how to evaluate materials for bias, 
but more work would need to be done to make it accessible as an evaluative instrument. It could also be 
used to inform the creation of a concise instrument. 

Tool Release Date: 2000 

Citation: Chu, C. (2000, Spring). Hear, and speak no evil: A content approach to evaluating multicultural 
multimedia materials. RUSQ: Reference and User Services Quarterly, 39(3), 255-64. 
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Table 1. Highlights from Tools and Resources 


Tool Name 


| Culturally Responsive Curriculum 
Scorecard 


| Assessing Bias in Standards and 
Curricular Materials 


Washington Models for the Evaluation of | 


Bias Content in Instructional Materials 


| Evaluating American Indian Materials 
and Resources for the Classroom 


Edgenuity: Curriculum Development 
Guidelines on Bias and Sensitivity 


| Bias Evaluation Instrument 


Fairness, Bias, and Cultural- 
Responsiveness Checklist for 
Assessments 


Highlights: 


DXWide range of forms of bias 

XXIIn-depth coverage of identities/intersectionalities 
XDifferent dimensions or domains to assess 
DXDefinition of instructional materials provided 


| DXWide range of forms of bias 


[ In-depth coverage of identities /intersectionalities 
Kx|Different dimensions or domains to assess 
[Definition of instructional materials provided 


[_]Wide range of forms of bias 

[_]In-depth coverage of identities /intersectionalities 
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L_|Definition of instructional materials provided 
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DXDefinition of instructional materials provided 


| [ ]Wide range of forms of bias 
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[ Different dimensions or domains to assess 
[Definition of instructional materials provided 


Kx]Wide range of forms of bias 
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Conclusion: Key Considerations 


As local education agencies and schools consider engaging in the process of evaluating tools for assessing bias in 
instructional materials, it is recommended that the following considerations be taken into account. These themes 
provide insight into not just the evaluation process but also can guide professional learning for educators seeking 
to eliminate bias in instructional materials. 


Wide range of forms of bias: Bias exists in forms of invisibility, tokenism, participation, privilege, representation, 
and outcomes. Initial reviews of curriculum and instructional materials should not only consider federally 
protected classes of immutable traits including race, color, national origin, religion, sex/gender, age, and dis/ability 
but also extend to include other forms. Bias can be present in content regarding language & linguistics, 
socioeconomic status, familial status, sexuality, citizenship, indigeneity, body size, content formatting, careers, 
geographic region, and other social and cultural perspectives (e.g., collectivism; individualism) and can also exist 
when considerations are not made regarding the impact of intersecting identities that make experiences unique 
(e.g., a Black female immigrant’s experiences taking STEM courses in a rural setting). Finally, omission of or 
perspectives regarding the impact of history (e.g., genocide, colonization, perspectives of war) contributes to the 
pervasiveness of institutional and cultural systems of power. Supporting instructional teams to understand the 
complexity of bias is critical to the use of tools and guidance and to the review of curriculum materials and teacher 
resources. 


In-depth coverage of identities /intersectionalities: Besides considering the range of various forms of bias, 
curriculum evaluation tools were reviewed for how in-depth the tool covered multiple and intersecting identities. 
As outlined above, intersectionality considers how specific identities combine to create unique experiences and 
influence the complexity of oppression and privilege one may face. Some resources were more thorough in the 
details or depths of bias present in curriculum. 


Different dimensions or domains to assess: Resources were reviewed to determine if they included dimensions 
to be evaluated or quantified using a rubric or scale measuring various constructs of equity or bias. 


Definition of instructional materials provided: The evaluation resources may have included directions on how 
they use the instrument, while also including definitions of the variety of instructional materials considered. 


Includes examples/broad explanations with criteria: In addition to a wide range of forms of bias and in-depth 
coverage of identities/intersectionalities, tools and resources were reviewed for their inclusion of examples of how 
instructional materials could be enhanced to eliminate bias. Several of the tools and resources shared provide 
before and after examples of language used in curriculum. 


Requires deep professional learning to support use: While resources may have included guidance on how to 
use them, some were also extensive in length, depth, or sophistication, therefore requiring the need for additional 
professional learning or assistance to support evaluation of curriculum with fidelity. 


Contextualized to a particular region: The review found that specific tools may have been made for a particular 
audience or based on regional details; however, those tools included were still useful for generalizing to broad 
aspects of bias in curriculum. 
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